Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of Allow dialer to re-establish terminated peering into release/1.14.x #16865

Conversation

hc-github-team-consul-core
Copy link
Collaborator

Backport

This PR is auto-generated from #16776 to be assessed for backporting due to the inclusion of the label backport/1.14.

The below text is copied from the body of the original PR.


Description

Currently, if an acceptor peer deletes a peering the dialer's peering will eventually get to a "terminated" state. If the two clusters need to be re-peered the acceptor will re-generate the token but the dialer will encounter this error on the call to establish:

failed to get addresses to dial peer: failed to refresh peer server addresses, will continue to use initial addresses: there is no active peering for "<<>>"

This is because in exchangeSecret().GetDialAddresses() we will get an error if fetching addresses for an inactive peering. The peering shows up as inactive at this point because of the existing terminated state.

Rather than checking whether a peering is active we can instead check whether it was deleted. This way users do not need to delete terminated peerings in the dialing cluster before re-establishing them.

Testing & Reproduction steps

  • The TestLeader_PeeringSync_Lifecycle_ServerDeletion test updated below shows how to reproduce the issue. It fails without this update.

PR Checklist

  • updated test coverage
  • external facing docs updated
  • not a security concern

Overview of commits

Andrew Stucki and others added 30 commits February 13, 2023 14:18
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* [API Gateway] Add integration test for HTTP routes
* Update index.mdx

* Update website/content/docs/connect/dataplane/index.mdx

Co-authored-by: Tu Nguyen <[email protected]>

---------

Co-authored-by: Tu Nguyen <[email protected]>
* chore: update dev build to 1.16

* chore(ci): add nightly 1.15 test
* draft

* expose internal admin port and add proxy test

* update tests

* move comment

* add failure case, fix lint issues

* cleanup

* handle error

* revert changes to service interface

* address review comments

* fix merge conflict

* merge the tests so cluster is created once

* fix other test
…to 1.24.2, add 1.25.1, remove 1.21.5 (#16274)

* Bump Envoy 1.22.5 to 1.22.7, 1.23.2 to 1.23.4, 1.24.0 to 1.24.2, add 1.25.1, remove 1.21.5
Prior to this PR, servers / agents would panic and crash if an ingress
or api gateway were configured to use a discovery chain that both:

1. Referenced a peered service
2. Had a mesh gateway mode of local

This could occur, because code for handling upstream watches was shared
between both connect-proxy and the gateways. As a short-term fix, this
PR ensures that the maps are always initialized for these gateway services.

This PR also wraps the proxycfg execution and service
registration calls with recover statements to ensure that future issues
like this do not put the server into an unrecoverable state.
* Fix infinite recursion on InlineCertificateConfigEntry

GetNamespace() + GetMeta() were calling themselves. This change also simplifies by removing nil-checking to match pre-existing config entries

Co-Authored-By: Andrew Stucki <[email protected]>

* Add tests for inline-certificate

* Add alias for private key field on inline-certificate

* Use valid certificate + private key for inline-certificate tests

---------

Co-authored-by: Andrew Stucki <[email protected]>
* for tab testing

* updates

* Update

* adding sandbox to test conf ref types

* testing tweaks to the conf ref template

* reintroduce tabbed specification

* applied feedback from MKO session

* applied feedback on format from luke and jared

* Apply suggestions from code review

Co-authored-by: Dan Upton <[email protected]>

* fixed some minor HCL formatting in complete conf

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <[email protected]>

* fixed bad link

* resolving conflicts

---------

Co-authored-by: boruszak <[email protected]>
Co-authored-by: Dan Upton <[email protected]>
Co-authored-by: Jeff Boruszak <[email protected]>
Fix mesh gateways incorrectly matching peer locality.

This fixes an issue where local mesh gateways use an
incorrect address when attempting to forward traffic to a
peered datacenter. Prior to this change it would use the
lan address instead of the wan if the locality matched. This
should never be done for peering, since we must route all
traffic through the remote mesh gateway.
* Simple API Gateway e2e test for tcp routes

* Drop DNSSans since we don't front the Gateway with a leaf cert

* WIP listener tests for api-gateway

* Return early if no routes

* Add back in leaf cert to testing

* Fix merge conflicts

* Re-add kind to setup

* Fix iteration over listener upstreams

* New tcp listener test

* Add tests for API Gateway with TCP and HTTP routes

* Move zero-route check back

* Drop generateIngressDNSSANs

* Check for chains not routes

---------

Co-authored-by: Andrew Stucki <[email protected]>
* Include secret type when building resources from config snapshot

* First pass at generating envoy secrets from api-gateway snapshot

* Update comments for xDS update order

* Add secret type + corresponding golden files to existing tests

* Initialize test helpers for testing api-gateway resource generation

* Generate golden files for new api-gateway xDS resource test

* Support ADS for TLS certificates on api-gateway

* Configure TLS on api-gateway listeners

* Inline TLS cert code

* update tests

* Add SNI support so we can have multiple certificates

* Remove commented out section from helper

* regen deep-copy

* Add tcp tls test

---------

Co-authored-by: Nathan Coleman <[email protected]>
* Fix hostname alignment checks for HTTPRoutes
* Add defensive guard to make some tests less flaky and panic less

* Do the actual fix
…onfigEntries (#16304)

* Add stricter validation and some normalization code for API Gateway ConfigEntries
… inline text for instructions (#16313)

* Update bug_report.md
* Update feature_request.md
* Update ui_issues.md
* Update pull_request_template.md
* security: update go to 1.20.1
Protobuf Refactoring for Multi-Module Cleanliness

This commit includes the following:

Moves all packages that were within proto/ to proto/private
Rewrites imports to account for the packages being moved
Adds in buf.work.yaml to enable buf workspaces
Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml
Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes)

Why:

In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage.
There were some recent changes to have our own ratelimiting annotations.
The two combined were not working when I was trying to use them together (attempting to rebase another branch)
Buf workspaces should be the solution to the problem
Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root.
This resulted in proto file name conflicts in the Go global protobuf type registry.
The solution to that was to add in a private/ directory into the path within the proto/ directory.
That then required rewriting all the imports.

Is this safe?

AFAICT yes
The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc)
Other than imports, there were no changes to any generated code as a result of this.
* new docs for consul and consul-k8s troubleshoot command

* add changelog

* add troubleshoot command

* address comments, and update cli output to match

* revert changes to troubleshoot upstreams, changes will happen in separate pr

* Update .changelog/16284.txt

Co-authored-by: Nitya Dhanushkodi <[email protected]>

* address comments

* update trouble proxy output

* add missing s, add required fields in usage

---------

Co-authored-by: Nitya Dhanushkodi <[email protected]>
* fix: add tls config to unix socket when https is used

* unit test and changelog
@hc-github-team-consul-core hc-github-team-consul-core requested a review from a team as a code owner April 3, 2023 22:26
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/peering/re-establish-terminated/immensely-active-beetle branch from 49f336d to 62e7b40 Compare April 3, 2023 22:26
@hc-github-team-consul-core hc-github-team-consul-core requested review from marianoasselborn and sarahethompson and removed request for a team April 3, 2023 22:26
@hc-github-team-consul-core hc-github-team-consul-core enabled auto-merge (squash) April 3, 2023 22:26
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/peering/re-establish-terminated/immensely-active-beetle branch from f9083cf to 779e28d Compare April 3, 2023 22:26
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto approved Consul Bot automated PR

@github-actions github-actions bot added pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/agent-cache Agent Cache theme/api Relating to the HTTP API interface theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified labels Apr 3, 2023
@freddygv
Copy link
Contributor

freddygv commented Apr 3, 2023

Not sure where this commit history came from, closing

@freddygv freddygv closed this Apr 3, 2023
auto-merge was automatically disabled April 3, 2023 22:27

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/agent-cache Agent Cache theme/api Relating to the HTTP API interface theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified
Projects
None yet
Development

Successfully merging this pull request may close these issues.